Scope sensing in a light controlled environment

ABSTRACT

The disclosure extends to methods, systems, and computer program products for detecting whether an illumination source of an endoscope is in use (inside the body of a patient) versus not in use (outside the body of a patient). The disclosure relies on the fact that the working environment is lit solely by the endoscope and its components. Thus, communication between the illumination or light source controller and the imaging device, such as a surgical camera, is required. When the illumination or light source is turned off and the endoscope is outside the body, a sensor will detect ambient light alerting the illumination source controller that it is outside the body, which then keeps the illumination source off or at a low intensity level. Conversely, when the illumination source is turned off and the endoscope is inside the body, the sensor will not detect any light (or will detect only a very low level of light). Based on this logic, if the imaging device, such as a camera, knows that the light is off during a specific period of time the frame(s) from that time period can be analyzed and the level of light gathered in the frame(s) will show the scope location either inside or outside of the body.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a division of U.S. application Ser. No. 14/214,334,filed Mar. 14, 2014, and claims the benefit of U.S. ProvisionalApplication No. 61/791,685, filed Mar. 15, 2013, which are herebyincorporated herein by reference in their entireties, including but notlimited to those portions that specifically appear hereinafter, theincorporation by reference being made with the following exception: Inthe event that any portion of the above-referenced applications isinconsistent with this application, this application supersedes saidabove-referenced applications.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable.

BACKGROUND

Current minimally invasive surgical procedures rely on endoscopes forvisualization of the surgical site. In the arthroscopy, laparoscopy,urology, gynecology, and ENT (ear, nose, and throat) specialties, rigidendoscopes are primarily used. A rigid endoscope is constructed of aninner lumen containing multiple glass lens elements for visualizationand an outer lumen containing a bundle of fiber optic strands forcarrying light from a light source to the surgical site.

Conventional surgical light systems are very inefficient. From the lightengine, which is typically a metal halide bulb, halogen bulb, xenonbulb, or LED(s) (light emitting diode), to the surgical site overninety-five percent of the light is lost. These losses occur at multiplelocations, the first being at the optic placed in front of the lightengine to gather the light from a wide dispersion angle and focus itinto a collimated beam with a diameter small enough to transmit to afiber optic light cable. The second loss point is the junction of thefocusing optic and the aforementioned fiber optic light cable. The fiberoptic light cable is a bundle, typically with a diameter of fivemillimeters, of small fiber optic strands and measures one to threemeters in length. The third loss point is over the length of the fiberbundle due to the attenuation rate of the bulk fiber strands. The fiberoptic light cable transmits light from the light source to the endoscopein sterile field. The fourth loss point is the junction between thelight cable and the proximal end of the endoscope.

Due to the losses in the light transmission path, the light source mustgenerate a significant amount of light. This results in a significantamount of heat generated, particularly at each of the junction pointsand at the distal tip of the scope. The heat generated, specifically atthe distal scope tip and at the junction between the light cable andscope, can present a safety risk to the surgical patient. The heat issuch that if the scope is inadvertently rested on the patient for aperiod of time, a burn can occur. This is an issue with all conventionallight sources and every year a few such incidents occur and are reportedto the FDA (Food and Drug Administration).

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive implementations of the presentdisclosure are described with reference to the following figures,wherein like reference numerals refer to like parts throughout thevarious views unless otherwise specified. Advantages of the presentdisclosure will become better understood with regard to the followingdescription and accompanying drawings where:

FIG. 1 illustrates a flow chart of an example method for controllingpower to an endoscopic light source in a light controlled environmentaccording to one implementation;

FIG. 2 illustrates an imaging device located outside of a lightdeficient environment, such as a patient's body, wherein the lightsource is turned off due to the amount of ambient light present inaccordance with the system and method described herein;

FIG. 3 illustrates an imaging device located in a light deficientenvironment, such as a patient's body, wherein the light source isturned on due to the lack of ambient light present in accordance withthe system and method described herein;

FIG. 4 illustrates a light threshold value and comparison chart of anexample method according to one implementation;

FIG. 5 illustrates an interval in a constant light system and a pulsedlight system wherein a frame is captured for analysis in accordance withthe system and method described herein;

FIG. 6 illustrates a system for controlling power to an endoscopic lightsource in a light controlled environment according to oneimplementation;

FIGS. 7A and 7B illustrate a perspective view and a side view,respectively, of an implementation of a monolithic sensor having aplurality of pixel arrays for producing a three dimensional image inaccordance with the teachings and principles of the disclosure;

FIGS. 8A and 8B illustrate a perspective view and a side view,respectively, of an implementation of an imaging sensor built on aplurality of substrates, wherein a plurality of pixel columns formingthe pixel array are located on the first substrate and a plurality ofcircuit columns are located on a second substrate and showing anelectrical connection and communication between one column of pixels toits associated or corresponding column of circuitry; and

FIGS. 9A and 9B illustrate a perspective view and a side view,respectively, of an implementation of an imaging sensor having aplurality of pixel arrays for producing a three dimensional image,wherein the plurality of pixel arrays and the image sensor are built ona plurality of substrates.

DETAILED DESCRIPTION

The disclosure extends to methods, systems, and computer programproducts for detecting whether an endoscopic illumination or lightsource is in use (inside the body of a patient) versus not in use(outside the body of a patient). The methods, systems and computerprogram products rely on the fact that the working environment is litsolely by the endoscope and its components. Thus, communication betweenthe illumination or light source controller and the imaging device, suchas a surgical camera, is required. In the following description of thepresent disclosure, reference is made to the accompanying drawings,which form a part hereof, and in which is shown by way of illustrationspecific implementations in which the disclosure may be practiced. It isunderstood that other implementations may be utilized and structuralchanges may be made without departing from the scope of the presentdisclosure.

It must be noted that, as used in this specification and the appendedclaims, the singular forms “a,” “an,” and “the” include plural referentsunless the context clearly dictates otherwise.

As used herein, the terms “comprising,” “including,” “containing,”“characterized by,” and grammatical equivalents thereof are inclusive oropen-ended terms that do not exclude additional, unrecited elements ormethod steps.

Further, where appropriate, functions described herein can be performedin one or more of: hardware, software, firmware, digital components, oranalog components. For example, one or more application specificintegrated circuits (ASICs) can be programmed to carry out one or moreof the systems and procedures described herein. Certain terms are usedthroughout the following description and Claims to refer to particularsystem components. As one skilled in the art will appreciate, componentsmay be referred to by different names. This document does not intend todistinguish between components that differ in name, but not function.

For safety and power consumption reasons, what is needed are methods andsystems for detecting when an illumination or light source is in use andwhen it is not in use. As will be seen, the disclosure provides methodsand systems that can do this in an efficient and elegant manner.

Referring now to the figures, it will be appreciated that the disclosurerelates to a detection mechanism for operating the illumination sourcewhen an endoscope is in use (inside the body of a patient) versus not inuse (outside the body of a patient). The disclosure relies on the factthat the working environment is lit solely by the endoscope and itscomponents. Thus, communication between the illumination or light sourcecontroller and the imaging device, such as a surgical camera, isrequired.

For safety reasons it is preferable to have the light source off whilethe endoscope is not in use. This removes the risk of burning a patientif, for example, the user inadvertently leaves the endo scope resting onthe patient while performing other tasks. Every year there are reportedcases of patient burns resulting from such misuse of conventionalendoscopic video systems.

When the light is turned off and the endoscope is outside the body, thesensor will detect ambient light. Conversely, when the light is turnedoff and the endoscope is inside the body, the sensor will not detect anylight (or will detect only a very low level of light). Based on thislogic, if the camera knows that the light is off during a specificperiod of time the frame(s) from that time period can be analyzed andthe level of light gathered in the frame(s) will show the scopelocation.

Knowing the location of the scope (inside or outside the body) allowsthe system to keep the light source off while outside the body and onlyturn the light source on when the endoscope is put into the body foruse.

Alternately, the light source output intensity can be reduced to a low,safe level while the scope is outside the body and then increased to ahigh level when inside the body and in use. This implementation may bepreferred for usability reasons. Users who are not familiar with thesystem described herein may suspect a functional problem with the systemif the light source is completely off while the scope is not in use.

Referring now to FIG. 1, there is illustrated a method for controllingpower to an endoscopic light source in a light controlled environment.As illustrated in the figure, at 110 the method may comprise sending anelectric communication signal from an imaging device controller to alight source controller at a specified interval. It will be appreciatedthat the signal may be an instruction to turn off the light source for apredetermined sample period during which a single sample frame or aplurality of sample frames may be collected from an image sensor. At120, the imaging device controller may receive a communication or datafrom the image sensor. Data may be collected from the image sensor for asingle frame or a plurality of frames obtained during the predeterminedsample period. The data may be related to an amount of ambient lightreceived by the image sensor. At 130, the imaging device controller mayanalyze the sample frame or plurality of frames against a predeterminedor predefined light threshold value. Analyzing the single frame orplurality of frames obtained during the predetermined sample periodagainst a specified, predetermined threshold value is also illustratedin FIG. 4. The operation of the light source may be controlled based onthe data received from the image sensor.

At 140, a determination is made by the image device controller. If theambient light is above or below the predetermined light threshold value,then one of two processes may be followed. Specifically, if the measuredlight from the image sensor is determined to be below the predeterminedlight threshold value, then at 150 it is determined that the imagesensor is in a light deficient environment. When it is determined thatthe light source is in a light deficient environment, that determinationsignifies that the imaging device is in-use. At 152, the light sourceremains in an operable state, thereby providing light to the lightdeficient environment. At 154, the light source may be turned off for apredetermined sample period and the process starts over again.

At 140, if the measured light from the image sensor is determined to beabove the predetermined light threshold value, then at 160 it isdetermined that the image sensor is not in-use because it is outside ofa light deficient environment. In such a circumstance, at 162, the lightsource is turned off, thereby providing a safety mechanism forcontrolling power to the light source. It will be appreciated that inone implementation, at 164, the turned off state may be a complete powerdown of the light source. In another implementation, at 166, the turnedoff state may be a reduction in power to the light source, such that thelight source is only emitting a small amount of light energy. As notedpreviously, the method may include sampling at a plurality intervals,such as a second interval, to determine whether data received from theimage sensor regarding a single frame is above or below thepredetermined light threshold value.

Referring now to FIGS. 2 and 3, it will be appreciated that in oneimplementation, the default mode when the endoscopic light source systemstarts-up is for the light source to be turned off or in an off state.At this time, the imaging device controller is in communication with thelight source controller and knows the light source is turned off or inan off state. At specified intervals, such as for example every 30^(th)frame, the imaging device captures a single frame and analyzes the lightlevel against the predetermined light threshold value. It is to beunderstood that the specified intervals may be at any frequency thatproduces the desired functional result. Further, it is to be understoodthat the light threshold value may be defined as an amount of totallight received by the image sensor; or the threshold value may bedefined as an average of an amount of light received per pixel on theimage sensor.

If the light source is determined to be not in-use as illustrated inFIG. 2, then the imaging device controller communicates information,instructions or data to the light source controller to remain off or inan off state. Conversely, if the light source is determined to be in-useas illustrated in FIG. 3, then the imaging device controllercommunicates information, instructions or data to the light sourcecontroller to turn on. Once the light source is turned on, a new patternbegins. Thus, at predetermined, specified intervals the light source isturned off for a predetermined sample period during which time theimaging device captures a single frame and analyzes the light levelagainst the predetermined threshold value. It is understood that thesample period may be any length that is long enough for the imagingdevice to capture one frame, but short enough that is does notnegatively affect video quality or user experience. As illustrated bestin FIG. 4, if the data received from the image sensor is below thepredetermined threshold value, then the imaging device recognizes thelight source as being in-use in a light deficient environment, and theimaging device controller communicates with the light source controllerto turn on. Whereas, if the data received from the image sensor is abovethe predetermined threshold value, then the image device recognizes thelight source as being not-in-use and is outside the light deficientenvironment, and the imaging device controller communicates with thelight source controller to remain off.

Referring now to FIG. 5, the light source may be a pulsed light system.In an implementation, the light in the pulsed light system may beobtained from laser light. In an implementation, the light in the pulsedlight system may be obtained from one or more light emitting diodes. Inanother implementation, the light source may be a constant light system.

It will be appreciated that the sampling interval may be every 30^(th)frame as described above, or it may be any other frequency that providesthe desired results. It is within the scope of the disclosure for theinterval frequency may be different during the “in-use” condition andthe “not-in-use” condition.

In an implementation, the imaging device, such as a camera, may provideconstant control over the light source. In an implementation, the lightsource may have a default state that is changed by the imaging device asrequired.

The method and system of the disclosure may require communicationbetween the light source controller and the imaging device controller.The disclosure also contemplates use of a light source with a responsetime that is fast enough that the “off” pulse during the sample period,during the “in-use” condition, does not adversely affect the videoquality. LED and laser light sources may be used, while a metal halidebulb, halogen bulb, or xenon bulb may not be used in thisimplementation.

During use, the light source can be kept on constantly with a periodic“off” pulse or the light source can be pulsed “on” during normal use,illustrated best in FIG. 5, with an “on” pulse skipped for the blackframe analysis.

In an implementation, the light intensity level can be reduced to apredetermined safe level while in the “not-in-use” state. In thisimplementation the default mode on startup could be a low lightintensity level that poses no risk of burning. Then, as previouslydescribed, at predetermined intervals the light is turned off for thesample period and this sample frame is analyzed. If the result is“not-in-use”, the light is turned back on at the previous safe level andthe pattern repeats. If the result is “in-use”, the light is turned onat the higher functional level.

In an implementation, the light could be pulsed light of a particularcolors (including, but not limited to, RBG or YCbCr) rather than whitelight. In this implementation it may be desirable to change from pulsedcolored light while “in-use” to pulsed or constant white light while“not-in-use” using the same techniques previously described. The defaultmode on startup could be a low level of pulsed or constant white light.Then, as previously described, at predetermined intervals the light isturned off for the sample period and this sample frame is analyzed. Ifthe result is “not-in-use”, the white light is turned back on at theprevious safe level and the pattern repeats. If the result is “in-use”,the pulsed color pattern is initiated.

In an implementation, the system may be comprised of a light source thatis kept in a constant on-state with a mechanical shutter providing theperiodic black frame. This shutter may be controlled by the imagingdevice, such that there would be no imaging device control of the lightsource needed. This shutter could be placed at any interface in thelight path from the source to the distal tip of the endoscope. In thisimplementation there is no restriction on light source technologybecause there is no requirement for the light source to have a fastresponse time. Instead, the mechanical shutter requires a response timethat is fast enough that the “off” pulse during the sample period,during the “in-use” condition, does not adversely affect the videoquality.

In any implementation, a visual or audible signal could be given toinform the user of whether the system is in the “in-use” or “not-in-use”state. Alternately, the signal could inform the user when the statechanges from “in-use” to “not-in-use” or from “not-in-use” to “in-use”or both.

A black frame would disrupt the video output. During image processing,the black frame can be removed and the previous frame can be displayedin its place. Conversely, multiple frames before and/or after the blackframe can be used to construct a substitute frame.

Referring now to FIG. 6, a system for controlling power to an endoscopiclight source in a light controlled environment is illustrated. Thesystem may comprise an imaging device 200 comprising an imaging devicecontroller 220, a light source comprising a light source controller 230;and an image sensor 240. It will be appreciated that the imaging devicecontroller may cause the system to perform the following processes: sendan electric communication signal to a light source controller at aspecified interval; turn off the light source for a predetermined sampleperiod based on the electric communication signal; collect data from theimage sensor for a single frame obtained during the predetermined sampleperiod, wherein the data relates to an amount of ambient light receivedby the image sensor; analyze the single frame obtained during thepredetermined sample period against a specified, predetermined thresholdvalue; and control the operation of the light source based on the datareceived from the image sensor.

Referring now to FIGS. 7A and 7B, the figures illustrate a perspectiveview and a side view, respectively, of an implementation of a monolithicsensor 700 having a plurality of pixel arrays for producing a threedimensional image in accordance with the teachings and principles of thedisclosure. Such an implementation may be desirable for threedimensional image capture, wherein the two pixel arrays 702 and 704 maybe offset during use. In another implementation, a first pixel array 702and a second pixel array 704 may be dedicated to receiving apredetermined range of wave lengths of electromagnetic radiation,wherein the first pixel array 702 is dedicated to a different range ofwave length electromagnetic radiation than the second pixel array 704.

FIGS. 8A and 8B illustrate a perspective view and a side view,respectively, of an implementation of an imaging sensor 800 built on aplurality of substrates. As illustrated, a plurality of pixel columns804 forming the pixel array are located on the first substrate 802 and aplurality of circuit columns 808 are located on a second substrate 806.Also illustrated in the figure are the electrical connection andcommunication between one column of pixels to its associated orcorresponding column of circuitry. In one implementation, an imagesensor, which might otherwise be manufactured with its pixel array andsupporting circuitry on a single, monolithic substrate/chip, may havethe pixel array separated from all or a majority of the supportingcircuitry. The disclosure may use at least two substrates/chips, whichwill be stacked together using three-dimensional stacking technology.The first 802 of the two substrates/chips may be processed using animage CMOS process. The first substrate/chip 802 may be comprised eitherof a pixel array exclusively or a pixel array surrounded by limitedcircuitry. The second or subsequent substrate/chip 806 may be processedusing any process, and does not have to be from an image CMOS process.The second substrate/chip 806 may be, but is not limited to, a highlydense digital process in order to integrate a variety and number offunctions in a very limited space or area on the substrate/chip, or amixed-mode or analog process in order to integrate for example preciseanalog functions, or a RF process in order to implement wirelesscapability, or MEMS (Micro-Electro-Mechanical Systems) in order tointegrate MEMS devices. The image CMOS substrate/chip 802 may be stackedwith the second or subsequent substrate/chip 806 using anythree-dimensional technique. The second substrate/chip 806 may supportmost, or a majority, of the circuitry that would have otherwise beenimplemented in the first image CMOS chip 802 (if implemented on amonolithic substrate/chip) as peripheral circuits and therefore haveincreased the overall system area while keeping the pixel array sizeconstant and optimized to the fullest extent possible. The electricalconnection between the two substrates/chips may be done throughinterconnects 803 and 805, which may be wirebonds, bump and/or TSV(Through Silicon Via).

FIGS. 9A and 9B illustrate a perspective view and a side view,respectively, of an implementation of an imaging sensor 900 having aplurality of pixel arrays for producing a three dimensional image. Thethree dimensional image sensor may be built on a plurality of substratesand may comprise the plurality of pixel arrays and other associatedcircuitry, wherein a plurality of pixel columns 904 a forming the firstpixel array and a plurality of pixel columns 904 b forming a secondpixel array are located on respective substrates 902 a and 902 b,respectively, and a plurality of circuit columns 908 a and 908 b arelocated on a separate substrate 906. Also illustrated are the electricalconnections and communications between columns of pixels to associatedor corresponding column of circuitry.

It will be appreciated that the teachings and principles of thedisclosure may be used in a reusable device platform, a limited usedevice platform, a re-posable use device platform, or asingle-use/disposable device platform without departing from the scopeof the disclosure. It will be appreciated that in a re-usable deviceplatform an end-user is responsible for cleaning and sterilization ofthe device. In a limited use device platform the device can be used forsome specified amount of times before becoming inoperable. Typical newdevice is delivered sterile with additional uses requiring the end-userto clean and sterilize before additional uses. In a re-posable usedevice platform a third-party may reprocess the device (e.g., cleans,packages and sterilizes) a single-use device for additional uses at alower cost than a new unit. In a single-use/disposable device platform adevice is provided sterile to the operating room and used only oncebefore being disposed of.

Additionally, the teachings and principles of the disclosure may includeany and all wavelengths of electromagnetic energy, including the visibleand non-visible spectrums, such as infrared (IR), ultraviolet (UV), andX-ray.

The foregoing description has been presented for the purposes ofillustration and description. It is not intended to be exhaustive or tolimit the disclosure to the precise form disclosed. Many modificationsand variations are possible in light of the above teaching. Further, itshould be noted that any or all of the aforementioned alternateimplementations may be used in any combination desired to formadditional hybrid implementations of the disclosure.

Further, although specific implementations of the disclosure have beendescribed and illustrated, the disclosure is not to be limited to thespecific forms or arrangements of parts so described and illustrated.The scope of the disclosure is to be defined by the claims appendedhereto, any future claims submitted here and in different applications,and their equivalents.

What is claimed is:
 1. A system for controlling power to an endoscopiclight source in a light controlled environment comprising: an imagingdevice comprising an imaging device controller; a light sourcecomprising a light source controller; an image sensor; wherein theimaging device controller causes the system to perform the followingprocesses: send an electric communication signal to a light sourcecontroller at a specified interval; turn off the light source for apredetermined sample period based on the electric communication signal,the predetermined sample period having a length sufficient for the imagesensor to capture at least a single frame; collect data from the imagesensor for the single frame obtained during the predetermined sampleperiod, wherein the data relates to an amount of ambient light receivedby the image sensor; analyze the single frame obtained during thepredetermined sample period against a specified, predetermined thresholdvalue; and control the operation of the light source based on the datareceived from the image sensor.
 2. The system of claim 1, wherein if thedata received from the image sensor is below the predetermined thresholdvalue, then the light source is recognized as being in-use by theimaging device controller and is in a light deficient environment. 3.The system of claim 2, wherein the light source remains in an operablestate to thereby provide light to the light deficient environment whenthe data received from the image sensor is below the predeterminedthreshold value.
 4. The system of claim 1, wherein if the data receivedfrom the image sensor is above the predetermined threshold value, thenthe light source is recognized as being not in-use by the imaging devicecontroller and is outside of a light deficient environment.
 5. Thesystem of claim 4, wherein the light source is turned off, therebyproviding a safety mechanism for controlling power to the light source.6. The system of claim 1, wherein a default mode when the endoscopiclight source system starts up is for the light source to be turned on.7. The system of claim 6, wherein at predetermined, specified intervalsthe imaging device controller communicates with the light sourcecontroller to turn off the light source for a predetermined sampleperiod, the imaging device captures a single frame and analyzes thelight level against the predetermined threshold value.
 8. The system ofclaim 7, wherein if the data received from the image sensor is below thepredetermined threshold value, then the imaging device recognizes thelight source as being in-use in a light deficient environment, and theimaging device controller communicates with the light source controllerto return to the default mode.
 9. The system of claim 7, wherein if thedata received from the image sensor is above the predetermined thresholdvalue, then the imaging device recognizes the light source as beingnot-in-use and is outside the light deficient environment, and theimaging device controller communicates with the light source controllerto remain off.
 10. The system of claim 1, wherein a default mode whenthe endoscopic light source system starts up is for the light source tobe turned off.
 11. The system of claim 10, wherein the imaging devicecontroller is in communication with the light source controller.
 12. Thesystem of claim 11, wherein at specified intervals the imaging devicecaptures the single frame and analyzes the light level against thepredetermined threshold value.
 13. The system of claim 12, wherein thepredetermined threshold value is an amount of total light received bythe image sensor.
 14. The system of claim 12, wherein the predeterminedthreshold value is an average of an amount of light received per pixel.15. The system of claim 10, wherein if the light source is determined tobe not in-use then the imaging device controller communicates to thelight source controller to remain off.
 16. The system of claim 10,wherein if the light source is determined to be in-use then the imagingdevice controller communicates to the light source controller to turnon.
 17. The system of claim 16, wherein once the light source is turnedon, a light pulsing pattern begins, such that at predetermined,specified intervals the light source is turned off for a predeterminedsample period, the imaging device captures a single frame and analyzesthe light level against the predetermined threshold value.
 18. Thesystem of claim 17, wherein if the data received from the image sensoris below the predetermined threshold value, then the imaging devicerecognizes the light source as being in-use in a light deficientenvironment, and the imaging device controller communicates with thelight source controller to turn on.
 19. The system of claim 17, whereinif the data received from the image sensor is above the predeterminedthreshold value, then the imaging device recognizes the light source asbeing not-in-use and is outside the light deficient environment, and theimaging device controller communicates with the light source controllerto remain off.
 20. The system of claim 1, wherein the imaging devicecontroller communicating with the system provides a user withinformation regarding the current state of the system.
 21. The system ofclaim 20, wherein the information is provided visually.
 22. The systemof claim 20, wherein the information is provided audibly.
 23. The systemof claim 1, wherein the imaging device controller communicating with thesystem provides a user with information regarding a change in the stateof the system.
 24. The system of claim 23, wherein the information isprovided visually.
 25. The system of claim 23, wherein the informationis provided audibly.